Introduction
A recent New York case shed light on the growing complexities of AI in expert witness testimony. An expert witness used an AI agent to estimate damages in a real estate dispute—only to have the court challenge the reliability of both the AI and the damages calculations in the judge’s order.
The court challenged the reliability of both the AI and the damages calculations in the judge’s order, issuing a stern warning about the need for disclosure before any AI-generated evidence is admitted. This incident serves as a cautionary tale for expert witnesses and attorneys navigating the uncharted waters of AI in legal proceedings.
I learned about this case from an Ars Technica article by Ashley Belanger. I highly recommend you read the article, “Expert witness used Copilot to make up fake damages, irking judge,” in addition to my description of the case below..
Trust & Estates Expert, Not Real Estate:
The expert in this case specializes in trust and estate litigation. More specifically, the trust and estates expert is knowledgeable in fiduciary duties, trustee standards of care, and prudent investor rules. The expert witness used Microsoft’s Copilot AI chatbot to assist in the damage calculations. This poor decision backfired when the court scrutinized the reliability of the AI-generated evidence and the expert’s methodology.
The judge’s order emphasized AI’s evolution in legal proceedings along with its inherent reliability issues. The court made it clear that before any AI-generated evidence can be admitted, full disclosure is critical. This incident is not an isolated one; there have been several notable cases where attorneys have submitted briefs citing non-existent case law generated by AI. These incorrect or wrongly generated case law citations are what you’ve heard described as hallucinations. It was only a matter of time before an expert witness made a similar misstep.
Keep within Your Expertise:
While the use of AI was problematic and rightly admonished by the court, it was not the primary issue in this case. The expert’s lack of relevant expertise in real estate appraisal, valuation, and damages was the root of the problem. As an expert witness, it is crucial to never opine outside your area of expertise. If a case falls outside your skillset, it is your responsibility to inform retaining counsel and either decline the assignment or limit your involvement to those aspects that are firmly within your expertise.
A close friend of mine and Experts.com, Mitch Jackson, Esq., seasoned lawyer and expert, recently declined an expert witness assignment we recently presented to him. He declined it for this very reason. It is better to turn down a case than to risk your reputation and the integrity of the proceedings by venturing into unfamiliar and inexpert territory.
The Dangers of Over-Reliance on AI:
In this case, the expert lacked the necessary expertise and relied on Microsoft Copilot to “cross-check” his calculations. Unfortunately, the AI-generated numbers were incorrect, further compounding the issue. This serves as a stark reminder that AI tools, while powerful, are not infallible and should be used with extreme caution.
If you don’t have relevant expertise, don’t know the proper formulas, and can’t verify the outputs, do not use the AI tool. Just as we used to check our math homework manually, the same principle applies to AI-generated results. If you cannot verify the accuracy of the output, it has no place in your expert report, deposition, or testimony.
The Reputational Risk:
Stories like this one, covered by reputable tech publications like Ars Technica, can have a devastating impact on an expert witness’s practice and reputation. In the digital age, news spreads quickly, and a single misstep can haunt an expert for years to come. It is crucial for experts to prioritize their credibility and integrity above all else, as their livelihood depends on it.
Best Practices for Expert Witnesses
To avoid falling into the same trap as the expert in this case, follow these best practices:
- Stay within your area of expertise: Only accept cases that fall squarely within your field of specialization.
- Verify your results: Always double-check your work and ensure that you can support your findings.
- Educate yourself on AI: As AI becomes more prevalent in legal proceedings, experts must stay informed about its capabilities and limitations.
- Prioritize your reputation: Your credibility is your most valuable asset as an expert witness. Protect it at all costs.
Best Practices for Attorneys:
- Vet experts thoroughly: Ensure that the expert has the necessary qualifications and expertise for all aspects of your case.
- Communicate expectations clearly: Discuss the scope of the expert’s work and inquire as to appropriate methods and tools.
- Review expert reports carefully: Scrutinize the expert’s findings and methodology, asking questions and raising concerns as needed.
- Stay informed about AI developments: Keep abreast of the latest advancements and legal precedents involving AI in expert testimony.
In Summation:
The case above serves as a stark reminder or a bold warning of the challenges and risks associated with uninformed usage of artificial intelligence in your expert witness practice. As AI continues to evolve and become more integrated into the legal practice and court proceedings, it is crucial for expert witnesses and attorneys to approach it with caution and diligence.
The Fun Stuff:
This article is an elaboration of a LinkedIn post I wrote approximately 1 month ago. To really bring the experience full circle, I pumped the text of the LinkedIn post into my chosen AI chatbot “LawDroid CoPilot.” So what you read above is a human-reviewed article written by AI. I did check my outputs, unlike the expert in the case above. Also, since it was my post to begin with, I was familiar enough with the content to be able to check my own work. Also, I had a teammate peer review before posting.
Now, I want you to be the judge. If you notice any glaring errors or mistakes in the article, please bring them to my attention so that we use this as a fun activity to improve our AI-assisted outputs. Hint, there are some phrases that are used more than once!
DISCLOSURE: LawDroid is a company owned and operated by my friend and colleague Thomas G. Martin. I’ve known Tom for nearly a decade through our efforts in the legal technology space. I just like to support my friends in their endeavors. The LawDroid CoPilot product was named CoPilot before Microsoft’s CoPilot.
